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A SHORTENED STATUTORY PERIOD FOR REPLY IS SET TO EXPIRE 3 MONTH(S) FROM 
THE MAILING DATE OF THIS COMMUNICATION. 

- Extensions of time may be available under the provisions of 37 CFR 1 .136(a). In no event, however, may a reply be timely filed 
after SIX (6) MONTHS from the mailing date of this communication. 
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earned patent term adjustment. See 37 CFR 1.704(b). 
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1 Responsive to communication(s) filed on 23 February 2005 . 
2a)S This action is FINAL. 2b)D This action is non-final. 

3) D Since this application is in condition for allowance except for formal matters, prosecution as to the merits is 

closed in accordance with the practice under Ex parte Quayle, 1935 CD. 11, 453 O.G. 213. 
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4) 0 Claim(s) 1 and 3-25 is/are pending in the application. 
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5) D Claim(s) is/are allowed. 

6) ^ Claim(s) 1 and 3-25 is/are rejected. 
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Application Papers 
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Replacement drawing sheet(s) including the correction is required if the drawing(s) is objected to. See 37 CFR 1.121(d). 
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Attachment(s) 

1) □ Notice of References Cited (PTO-892) 

2) Notice of Draftsperson's Patent Drawing Review (PTO-948) 

3) □ Information Disclosure Statement(s) (PTO-1 449 or PTO/SB/08) 

Paper No(s)/Mail Date . 



4) (ZI Interview Summary (PTO-413) 

Paper No(s)/Mail Date. . 

5) [Zl Notice of Informal Patent Application (PTO-1 52) 

6) □ Other: . 



U.S. Patent and Trademark Office 
PTOL-326 (Rev. 1-04) 



Office Action Summary 



Part of Paper No./Mail Date 20050814 



Application/Control Number: 09/769,599 
Art Unit: 2623 



Page 2 



Response to Arguments 



1. 



Applicant's amendment filed on 02/23/05 has been entered and made of 



record. 



2. 



Applicant has amended claims 1,11, 15, 16, and 22. Currently, claims 1- 



25 are pending. 



3. 



Applicant in essence argues that applicant's instant invention extracts 



motion information from an image sequence, then determines spatial patterns 
from the motion information and these spatial patterns represent trajectories of 
objects moving within the imaged scene and that neither reference of Courtney 
(U.S. patent 5,969,755) nor Gran et al. (U.S. patent 5,41 6,71 1 ) alone or in 
combination teaches the newly added limitation (claims 1, 1 1, 15, 16, and 22) of 
"the spatial patterns represent trajectories of objects in the sequence of images." 
Examiner disagrees. Examiner believes that the prior art of Courtney either alone 
or in combination with Gran et al. teaches this new limitation. Courtney teaches 
to obtain images of a scene then extract the motion from the images then 
perform object tracking on the images (Courtney; fig. 1, fig. 3, fig. 4, fig. 5 
elements 11, 21, 22, and 23, col. 3 lines 1-8, col. 4 lines 47-61, and col. 7 lines 
43-61). Here the system obtains the sequence of images of a scene performs the 
motion segmentation, a motion graph ("spatial pattern") of the motion determined 
(fig. 4, read as the "spatial pattern" since it is of a the motion of an object through 
a real 3D space which an example is shown in fig. 3 wherein the scene/3D space 
is a room) from which the object path/trajectory is determined (Courtney; col. 4 
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lines 47-61 and col. 7 lines 43-51 , wherein the path and trajectories are 
determined). Further, the prior art of Gran et al. teaches to observe objects 
moving through a corridor, which is read as a trajectory, since the objects are 
moving through this corridor and some of the moving objects move through the 
whole corridor, making this the path of these objects, while some others turn off 
this corridor (Gran et al.; fig. 6 and col. 8 lines 24-37). As defined by The 
American Heritage College Dictionary, fourth edition, a trajectory is "The path of 
a projectile or other moving body through space," therefore; examiner takes the 
corridor as equivalent to being a trajectory in this situation. As discussed above 
Gran et al. teaches to monitor moving objects through a corridor and which 
objects enter and exit this corridor and which ones turn off the corridor which is 
read as determining/detecting the paths/trajectories of the moving objects in the 
corridor. For the arguments made above examiner believes that Courtney and 
Gran et al. either alone or combined teach the newly added limitation in the 
claims discussed. Examiner refers to the rejection below. 



Claim Rejections - 35 USC § 103 

4. The following is a quotation of 35 U.S.C. 103(a) which forms the basis for 

all obviousness rejections set forth in this Office action: 

(a) A patent may not be obtained though the invention is not identically disclosed or described as set 
forth in section 102 of this title, if the differences between the subject matter sought to be patented and 
the prior art are such that the subject matter as a whole would have been obvious at the time thie 
invention was made to a person having ordinary skill in the art to which said subject matter pertains. 
Patentability shall not be negatived by the manner in which the invention was made. 
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A.) Claims 1, 4, 8, 1 1, 12, and 15-25 are rejected under 35 U.S.C. 103(a) 
as being unpatentable over Courtney (U.S. patent 5,969,755) and Gran et al. 
(U.S. patent 5,416,711). 

Regarding claims 1 and 16: Courtney discloses a method for performing 
motion analysis on a sequence of images (Courtney; figs. 2-4 and col. 3 lines 1- 
2), where said sequence of images captures a plurality of objects each moving 
along a trajectory in an imaged area (Courtney; figs. 9-12, col. 3 lines 1-8, and 
col. 7 lines 47-51), said method comprising: 

extracting motion information for each of said plurality of objects contained 
in said sequence of images (Courtney; col. 4 lines 29-31 and 54-61 , wherein the 
information is segmented from the images in order to obtain the stationary and 
moving objects), and 

determining spatial patterns from said extracted motion information, the 
spatial patterns represent trajectories of objects in the sequence of images 
(Courtney; fig. 1 , fig. 3, fig. 4, fig. 5 elements 1 1 , 21 , 22, and 23, col. 3 lines 1-8, 
col. 4 lines 47-61, and col. 7 lines 43-61, wherein the paths of the objects are 
determined from the segmented/extracted information. These paths are read as 
spatial patterns and as the routes of the objects, also see examiner's rebuttal 
above), where said 

determining step comprises determining a route comprising a trajectory of 
a first object having the same trajectory of at least one other object. 
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Courtney discloses to obtain moving and stationary objects within a 
video/sequence of images by determining the paths/routes/trajectories of the 
objects. Courtney does not disclose to obtain determining the "route comprising a 
trajectory of a first object having the same trajectory of at least one other object." 
Gran et al. teaches to determine the route of a first object having the same 
trajectory of at least one other object (Gran et al.; col. 2 lines 59-68 and col. 8 
lines 24-34, wherein a traffic corridor, read as a "trajectory" of the traffic, is 
monitored to observe the length of the traffic queue and/or the number of 
vehicles in the traffic queue. Determining the number of vehicles within the traffic 
queue/corridor is read as determining of "a trajectory of a first object having the 
same trajectory of at least one other object," since more than one object is 
determined in the same traffic corridor "trajectory."). It would have been obvious 
to one skilled in the art to combine the teaching of Gran et al. to that of Courtney 
because they are analogous in monitoring moving objects/traffic patterns 
(Courtney; col. 11 lines 1-4 and Gran et al.; col. 1 lines 7-10). One in the art 
would have been motivated to incorporate the teaching of Gran et al. to the 
system of Courtney in order to provide an efficient and safe utilization of the 
nation's roads and highway systems (Gran et al.; col. 1 lines 26-28). 

Regarding claims 4, 20, and 24: The method of claim 1 wherein said 
determining of spatial patterns comprises: 

determining a source point and a destination point from said trajectory of 
said plurality of objects (Courtney; col. 4 lines 62-67, wherein the entrance and 
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exit points of the moving and stationary objects are determined, which are read 
as source and destination points, respectively). 

Regarding claim 8: The method further comprising: 

determining spatio-temporal patterns from said determined spatial 
patterns along a time dimension (Courtney; col. 4 lines 7-1 1 , wherein the events 
are spatio-temporal). 

Regarding claims 1 1 and 12: They are rejected for the combination of 
reasons of claim 1,9, and 10. 

Regarding claim 15: It is rejected for the same reasons as claim 1 above 
and for the following limitation of a method for displaying motion information of 
objects contained in a sequence of images (Courtney; fig. 5 elements 19 and 27- 
29), the method comprising: 

performing a query on a plurality of spatial patterns stored in a database, 
where each of said plurality of spatial patterns comprises a route determined 
from a trajectory common to at least two objects moving in an imaged area 
captured in said sequence of images and displayed on a user interface 
(Courtney; fig. 5 elements 27-29 and col. 5 lines 12-20 and 43-51). 

Regarding claim 17. The system further comprising a video source for 
capturing said plurality of objects in an imaged area and transmitting video 
containing said captured plurality of objects to said motion extraction system 
(Courtney; fig. 5 elements 11, 15, and 21-23, wherein a video camera obtains 
images and the images are then analyzed for motion). 
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Regarding claim 18: The system further comprising: 

a database for storing said spatial patterns determined from said motion 
mining system (Courtney; fig. 5 element 15); and 

a server computer for retrieving said trajectory satisfying at least one 
constraint specified in a query (Courtney; fig. 5 elements 27 and 28). 

Regarding claims 19 and 23: It is rejected for the same reasons as claim 

1. 

Regarding claims 21 and 25: The system wherein said motion mining 
system determines spatio-temporal patterns from said spatial patterns along a 
time dimension (Courtney; col. 4 lines 7-1 1 , wherein the events are spatio- 
temporal). 

Regarding claim 22: It is rejected for the same reasons as claim 1 and for 
the following limitations of: 

a memory for storing a motion mining program (Courtney; fig. 5 element 

15); 

an interface (Courtney; fig. 5 elements 27 and 28); 
a processor (Courtney; fig. 5 elements 27 and 28). 

B.) Claim 3 is rejected under 35 U.S.C. 103(a) as being unpatentable over 
Courtney (U.S. patent 5,969,755), as modified by Gran et al. (U.S. patent 
5,416,71 1), and further in view of Koike (U.S. patent 6,445,308 B1). 
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Regarding claim 3: Courtney discloses to obtain moving and stationary 
objects within a video/sequence of images by determining the 
paths/routes/trajectories of the objects. Courtney, as modified by Gran et al., 
teaches to obtain a number of objects within the same trajectory. Neither 
Courtney nor Gran et al. teach to "determining whether said trajectory of a 
second object is within a threshold distance said trajectory of said first object 
and including, if said trajectory of said second object is within the threshold 
distance, said trajectory of said second object in said route." Koike teaches this 
limitation wherein the trajectory of the first object is determined to be within a 
threshold value of the trajectory of a second object (Koike; fig. 20 and col. 17 lines 
36-44 and 54-59, wherein a threshold is used to determine the possibility of a 
collision based on the objects trajectories, i.e. two moving object trajectories 
coming within a certain distance/threshold). It would have been obvious to one 
skilled in the art to combine the teaching of Koike to the disclosure of Courtney, 
as modified by Gran et al., because they are analogous in determine object 
motion path using trajectories. One in the art would have been motivated to 
incorporate the teaching of Koike to the system of Courtney, as modified by Gran 
et al., "to avert a collision with another vehicle by obtaining accurate motion 
information" (Koike; col. 1 lines 63-64). 

C.) Claims 5, 6, 9, 10, 13, and 14 are rejected under 35 U.S.C. 103(a) as 
being unpatentable over Courtney (U.S. patent 5,969,755), as modified by Gran 
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et al. (U.S. patent 5,416,711), and further in view of Krause et al. (EP 740280 
A2). 

Regarding claims 5, 6, and 13: Courtney discloses a system to determine 
multiple objects in motion within a video. Courtney further discloses to determine 
the objects entry and exit points (Courtney; col. 4 lines 62-67, wherein the entry 
and exit points are read as the source and destination points, respectively). 
Courtney, as modified by Gran et al., does not teach "determining whether a 
number of trajectories originating from a location is greater than a threshold 
number and identifying, if the number of trajectories originating from the location 
Is greater than the threshold number, the location as said source point" nor do 
they teach "determining whether a number of trajectories ending at a location is 
greater than a threshold number and identifying, if the number of trajectories 
ending at the location is greater than the threshold number, the location as said 
destination point." Krause et aL teaches to look at the number of cars coming into 
a sector/region and the number of cars at the end of the sector to determine 
against a threshold value if there is a hold up in that traffic sector (Krause et al.; 
English translated version of the abstract which has been provided). It would 
have been obvious to one skilled in the art to combine the teaching of Krause et 
al. to that of Courtney, as modified by Gran et al., because they are analogous in 
traffic monitoring. One in the art would have been motivated to incorporate the 
teaching of Krause et al., modified to threshold the trajectories for their number of 
entrance points and their exit points and label them accordingly, to the system of 
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Courtney, as modified by Gran et al., in order to determine the traffic level and 
speed (Krause et al.; abstract). 

Regarding claims 9 and 10: Courtney discloses a system wherein the 
motion of objects in a video are determined. Courtney further discloses to obtain 
the events as spatio-temporal information wherein the movements will be 
recorded according to the time that the event(s) happened (Courtney; col. 4 lines 
54-67 and col. 5 lines 1-17). Courtney does not teach the feature of "determining 
a busy time for said route, where the busy time represents a time when a number 
of trajectories for said plurality of objects along said route is greater than a 
threshold number" nor teaches "determining a periodicity of at least one 
.trajectory in a route." Krause et al. teaches to obtain the number of vehicles in a 
sector based in a sector of traffic to see if there is a traffic hold up (Krause et al.; 
abstract). It would have been obvious to one skilled in the art based on 
Courtney's disclosure and the teaching of Krause to modify the system wherein 
the events of object motion/trajectories can be determined to see when it is/are 
the busiest time(s), how often it is happening/periodicity/frequency, how many 
objects involved, etc. 

Regarding claim 14: It is rejected for the same combination of reasons of 
claim 1 with claims 9 and 10. 
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D.) Claim 7 is rejected under 35 U.S.C. 103(a) as being unpatentable over 
Courtney (U.S. patent 5,969,755), as modified by Gran et al. (U.S. patent 
5,416,71 1), and further in view of Auty et al. (U.S. patent 5,809,161). 

Regarding claim 7: Courtney discloses a system to determine multiple 
objects in motion within a video. Courtney further discloses to determine the 
objects entry and exit points (Courtney; col. 4 lines 62-67, wherein the entry and 
exit points are read as the source and destination points, respectively). Courtney, 
as modified by Gran et al., does not teach "wherein said source point and said 
destination point are determined using a clustering process". Auty et al. teaches 
to use cluster processing to obtain and classify certain points in an images of 
moving objects (Auty et al.; col. 12 lines 65-67 and col. 20 lines 20-35). It would 
have been obvious to one skilled in the art to combine the teaching of Auty et al., 
modified to determine the source and destination points using clustering, to the 
system Courtney, as modified by Gran et al., for tracking moving objects over a 
multi-lane carriageway (Auty et al.; col. 2 lines 53-55). 

Conclusion 

5. Applicant's amendment necessitated the new ground(s) of rejection 

presented in this Office action. Accordingly, THIS ACTION IS MADE FINAL. 
See MPEP § 706.07(a). Applicant is reminded of the extension of time policy as 
set forth in 37 CFR 1.136(a). 
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A shortened statutory period for reply to this final action is set to expire 
THREE MONTHS from the mailing date of this action. In the event a first reply is 
filed within TWO MONTHS of the mailing date of this final action and the advisory 
action is not mailed until after the end of the THREE-MONTH shortened statutory 
period, then the shortened statutory period will expire on the date the advisory 
action is mailed, and any extension fee pursuant to 37 CFR 1 .136(a) will be 
calculated from the mailing date of the advisory action. In no event, however, will 
the statutory period for reply expire later than SIX MONTHS from the date of this 
final action. 

Contact Information 

6. Any inquiry concerning this communication or earlier communications from 

the examiner should be directed to Anand Bhatnagar whose telephone number is 
(571) 272-7416, whose acting supervisor is Jingge Wu whose number is (571) 
272-7429, Central fax is 571-273-8300, and Tech center 2600 customer service 
office number is 703-306-0377. 
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